In the Diabetes Control and Complication Trial (DCCT) (1), the risks of microvascular (retinopathy and nephropathy) and neuropathic complications of diabetes were predicted by Hemoglobin A1c (HbA1c) for subjects with Type 1 diabetes (T1 DM). Similarly, for those with Type 2 diabetes (T2DM), in the United Kingdom Prospective Diabetes Study (UKPDS) the risks of microvascular complications of diabetes were predicted based on HbA1c (2). However, blood glucose (BG) fluctuations are a process in time and understanding their temporal characteristics is important. Despite the excellent predictive power of HbA1c for chronic diabetes complications, patients commonly experience disturbing symptoms related to rapid blood glucose (BG) swings deviating from their average glycemia. Measures of average glycemia thus may fail to capture important aspects of the temporal patterns of glycemia (3). Recently, it has been reported that extreme fluctuations in glucose values are important markers of diabetes control in both T1DM and T2DM (4): (i) the Low BG Index (LBGI) (5, 6) predicts the risk and frequency of severe hypoglycemia; (ii) the High BG Index (HBGI) predicts the risk of extremes of hyperglycemia (5, 7), and (iii) previously reported symptom-BG associations (8) and recent data (9) suggest that BG rate of change may be linked to the development of negative mood and cognitive symptoms, especially post meals (9). Mounting evidence point to the importance of rapid BG fluctuations for long-term diabetes complications as well. A number of recent studies found that postprandial hyperglycemia is an independent factor contributing to cardiovascular complications and increased mortality, especially in T2DM (10, 11, 12, 13, 14, 15). The Diabetes Intervention Study, concluded that postprandial, but not fasting BG, was an independent predictor of mortality in T2DM (15). A recent review of studies in this area concluded that there are now comprehensive and consistent data from pathophysiological as well as epidemiologic studies that excessive post-load glucose excursions have acute and chronic harmful effects on the endothelium and vessel wall (16). Thus, an accurate assessment of BG dynamics in peoples' natural environment is a valuable tool for both everyday maintenance of diabetes and long-term effectiveness of glycemic control. This is the premise behind increasing industrial and research efforts concentrated on the development of sensors for continuous, or nearly continuous, monitoring of BG (CGS, 17, 18, 19). Compared to a few self-monitoring BG (SMBG) readings per day, continuous glucose sensor (CGS) yield detailed time series of BG determinations, e.g. BG samples every 5 minutes for several days. The evaluation of the accuracy of CGS, however, is not straightforward, especially if taken in the context of established accuracy measures, such as statistical correlation or regression, or the clinically based Error-Grid Analysis (EGA) previously introduced (20, 21). A problem is that all these accuracy measures are designed to reflect the quality of approximation of reference BG by measurements taken in isolated static points in time, regardless of the temporal structure of the data. As such, these measures work well for evaluation of self-monitoring (SMBG) devices and are accepted by FDA as valid supplements to the review of clinical measurement methods (22, 23). However, applying these measures to evaluate the process approximation offered by CGS is questionable. An analogy of CGS vs. SMBG with camcorders vs. still cameras is inevitable, and might be helpful. Still cameras produce highly accurate snapshots of a process in time; camcorders generally offer lower resolution and precision of each separate image, but capture the dynamics of the action. Thus, it would be inappropriate to gauge the accuracy of still cameras and camcorders using the same static measure—the number of pixels in a single image. Similarly, it is inappropriate to gauge the precision of CGS and SMBG devices using the same measures, especially when these measures ignore the temporal characteristics of the observed process.
The original Error Grid Analysis (EGA) some time ago, which was intended to quantify the clinical significance of the agreement between a BG estimate and reference value (20). While other methods were available to quantify the statistical significance of such agreement, such as correlational and regression analyses, these approaches did not address the important question of the clinical implications of accuracy of BG estimates, i.e., what would be the potential outcome if a patient took some self-treatment action based upon a BG estimate. In its original form, the EGA was used to quantify the accuracy of patient BG estimates, based on physical symptoms and information about time of day, insulin, food and physical activity, compared to BG meter values. Subsequently, the EGA was used to assess the accuracy of BG meters compared to reference laboratory measurements (21). Over time, the EGA became one of the accepted standards for demonstrating acceptable levels of accuracy of BG meters to the FDA (23).
The original EGA plots each BG estimate (from any source) on the Y-axis against its companion reference BG measure on the X-axis, so that each estimate falls into one of 10 zones (upper and lower A-E zones). The upper and lower A zones are clinically accurate, indicating that a BG estimate deviates less than 20% from the reference, or that both the estimate and reference value are <70 mg/dl. These estimates are considered accurate because they are likely to lead to an appropriate clinical response (e.g., attempting to raise a BG value that is too low or lower a BG value that is too high). Clinically significant errors are those that fall into the upper and lower 1) C zones (possible over-correction when estimated BG indicates treatment may be needed but reference value is in an acceptable range), 2) D zones (failure to detect and possibly treat reference BG levels that are too low or too high, and 3) E zones (erroneous estimates indicating that BG needs to be raised when it is actually too high already, and vice versa). Estimates falling into the remaining area of the Error Grid are B zone errors, indicating benign errors that deviate more than 20% from the reference value but are unlikely to lead to any clinical action or have significant implications if action is taken.
Once the original EGA was published, the scientific community quickly recognized the important unique information provided by this method of evaluating accuracy, and it became a standard component of almost all clinical trials of new BG meters. This was an appropriate use of the EGA since BG meters are designed to provide an accurate estimate of a single static BG value. However, because the EGA only quantifies point accuracy of each single static estimate of BG, and does not take into consideration temporal characteristics of BG fluctuations, its use for determining the accuracy of CGS devices is problematic. In fact, because the EGA does not quantify the accuracy of estimations of temporal changes, such as rate and direction of BG fluctuation, it may even yield misleading results concerning the clinical implications of errors in CGS devices.
Traditional device evaluation methods fail to capture a most important temporal characteristic of the continuous glucose monitoring process.